add has_import_errors filter to Core API GET /dags endpoint#54563
add has_import_errors filter to Core API GET /dags endpoint#54563pierrejeambrun merged 13 commits intoapache:mainfrom
Conversation
There was a problem hiding this comment.
Nice, do you mind adding some basic tests related to this addition. There is already a parameterized test for that:
test_dags.py def test_get_dags.
Also can you update the similar private endpoint (+test), because the UI actually use the private API. (ui/dags get_dags endpoint)
|
Related Documentation No published documentation to review for changes on this repository. |
jason810496
left a comment
There was a problem hiding this comment.
Thanks for the PR! Same comment as Pierre.
FYI, here is the UI route to add the filter as well, thanks!
|
Thanks @pierrejeambrun and @jason810496 , PR updated. |
|
Copying the comment from the parent issue for reference:
|
jason810496
left a comment
There was a problem hiding this comment.
So I think it's still useful to create such filter. I also suggest to update any related doc,
to highlight the Import Errors filter only returns DAGs that have been successfully loaded before.
Agree, after Add description field for filter_param_factory #54903 we can add description for query parameter with the following code suggestion.
jason810496
left a comment
There was a problem hiding this comment.
Additionally, the conflict need to be resolved, thanks!
pierrejeambrun
left a comment
There was a problem hiding this comment.
NIce. Just one nit.
Good to merge once we rebased and solved conflicts.
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dags.py
Outdated
Show resolved
Hide resolved
|
Thanks @pierrejeambrun and @jason810496! I've addressed your comments, plz review again. |
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
…4563) * add has_import_errors filter * add unit tests * update test * add description * Update airflow-core/src/airflow/api_fastapi/common/parameters.py Change DAGs to Dags Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com> * empty commit * Change DAGs to dags * Change DAGs to dags --------- Co-authored-by: Guangyang Li <mail@guangyangli.com> Co-authored-by: LIU ZHE YOU <68415893+jason810496@users.noreply.github.com>
(from https://github.com/apache/airflow/tree/python-client/3.1.0rc1) ## New Features: - Add `map_index` filter to TaskInstance API queries ([#55614](apache/airflow#55614)) - Add `has_import_errors` filter to Core API GET /dags endpoint ([#54563](apache/airflow#54563)) - Add `dag_version` filter to get_dag_runs endpoint ([#54882](apache/airflow#54882)) - Implement pattern search for event log endpoint ([#55114](apache/airflow#55114)) - Add asset-based filtering support to DAG API endpoint ([#54263](apache/airflow#54263)) - Add Greater Than and Less Than range filters to DagRuns and Task Instance list ([#54302](apache/airflow#54302)) - Add `try_number` as filter to task instances ([#54695](apache/airflow#54695)) - Add filters to Browse XComs endpoint ([#54049](apache/airflow#54049)) - Add Filtering by DAG Bundle Name and Version to API routes ([#54004](apache/airflow#54004)) - Add search filter for DAG runs by triggering user name ([#53652](apache/airflow#53652)) - Enable multi sorting (AIP-84) ([#53408](apache/airflow#53408)) - Add `run_on_latest_version` support for backfill and clear operations ([#52177](apache/airflow#52177)) - Add `run_id_pattern` search for Dag Run API ([#52437](apache/airflow#52437)) - Add tracking of triggering user to Dag runs ([#51738](apache/airflow#51738)) - Expose DAG parsing duration in the API ([#54752](apache/airflow#54752)) ## New API Endpoints: - Add Human-in-the-Loop (HITL) endpoints for approval workflows ([#52868](apache/airflow#52868), [#53373](apache/airflow#53373), [#53376](apache/airflow#53376), [#53885](apache/airflow#53885), [#53923](apache/airflow#53923), [#54308](apache/airflow#54308), [#54310](apache/airflow#54310), [#54723](apache/airflow#54723), [#54773](apache/airflow#54773), [#55019](apache/airflow#55019), [#55463](apache/airflow#55463), [#55525](apache/airflow#55525), [#55535](apache/airflow#55535), [#55603](apache/airflow#55603), [#55776](apache/airflow#55776)) - Add endpoint to watch dag run until finish ([#51920](apache/airflow#51920)) - Add TI bulk actions endpoint ([#50443](apache/airflow#50443)) - Add Keycloak Refresh Token Endpoint ([#51657](apache/airflow#51657)) ## Deprecations: - Mark `DagDetailsResponse.concurrency` as deprecated ([#55150](apache/airflow#55150)) ## Bug Fixes: - Fix dag import error modal pagination ([#55719](apache/airflow#55719))
(from https://github.com/apache/airflow/tree/python-client/3.1.0rc1) ## New Features: - Add `map_index` filter to TaskInstance API queries ([#55614](apache/airflow#55614)) - Add `has_import_errors` filter to Core API GET /dags endpoint ([#54563](apache/airflow#54563)) - Add `dag_version` filter to get_dag_runs endpoint ([#54882](apache/airflow#54882)) - Implement pattern search for event log endpoint ([#55114](apache/airflow#55114)) - Add asset-based filtering support to DAG API endpoint ([#54263](apache/airflow#54263)) - Add Greater Than and Less Than range filters to DagRuns and Task Instance list ([#54302](apache/airflow#54302)) - Add `try_number` as filter to task instances ([#54695](apache/airflow#54695)) - Add filters to Browse XComs endpoint ([#54049](apache/airflow#54049)) - Add Filtering by DAG Bundle Name and Version to API routes ([#54004](apache/airflow#54004)) - Add search filter for DAG runs by triggering user name ([#53652](apache/airflow#53652)) - Enable multi sorting (AIP-84) ([#53408](apache/airflow#53408)) - Add `run_on_latest_version` support for backfill and clear operations ([#52177](apache/airflow#52177)) - Add `run_id_pattern` search for Dag Run API ([#52437](apache/airflow#52437)) - Add tracking of triggering user to Dag runs ([#51738](apache/airflow#51738)) - Expose DAG parsing duration in the API ([#54752](apache/airflow#54752)) ## New API Endpoints: - Add Human-in-the-Loop (HITL) endpoints for approval workflows ([#52868](apache/airflow#52868), [#53373](apache/airflow#53373), [#53376](apache/airflow#53376), [#53885](apache/airflow#53885), [#53923](apache/airflow#53923), [#54308](apache/airflow#54308), [#54310](apache/airflow#54310), [#54723](apache/airflow#54723), [#54773](apache/airflow#54773), [#55019](apache/airflow#55019), [#55463](apache/airflow#55463), [#55525](apache/airflow#55525), [#55535](apache/airflow#55535), [#55603](apache/airflow#55603), [#55776](apache/airflow#55776)) - Add endpoint to watch dag run until finish ([#51920](apache/airflow#51920)) - Add TI bulk actions endpoint ([#50443](apache/airflow#50443)) - Add Keycloak Refresh Token Endpoint ([#51657](apache/airflow#51657)) ## Deprecations: - Mark `DagDetailsResponse.concurrency` as deprecated ([#55150](apache/airflow#55150)) ## Bug Fixes: - Fix dag import error modal pagination ([#55719](apache/airflow#55719))
Resolves #53536